Face Registration with Active Appearance Models for Local Appearance-based Face Recognition

Face recognition has received increasing attention from diverse research communities and the market over the past years. Various techniques have been intensively investigated aiming at high recognition accuracy and robustness against numerous facial appearance variations. Application areas of face recognition have also been expanded and more robust systems are required as the application scenarios become more unconstrained. In this work, variation in facial appearance caused by 3D head pose was considered. The problem is also known as the face registration problem, which is an important factor for face recognition as demonstrated in many previous studies. The registration approach studied in this thesis is able to normalize the head pose in some degree of rotation in depth and align the face into a common coordinate framework. Moreover, the quality of face registration is assessed so that only successfully registered face images are used for recognition. The developed face registration approach is based on active appearance model (AAM) fitting. A generic model was built in which both shape and appearance variations were modeled. After fitting the model on an input image, the pose of the input face was normalized and a frontal view of the input face was synthesized. To mitigate the influence of poor illumination, a modified histogram fitting approach was employed. Progressive model fitting was also investigated for a more robust estimate of model initialization. Face recognition was based on the fitted and pose normalized face images using our local appearance-based approach. Three experiments were conducted to evaluate the AAM-based face registration approach. The first experiment was designed to evaluate the pose correction based on AAM fitting in still images. The results showed a significant improvement in face recognition performance compared to the previous affinebased registration approach, which again demonstrated the importance of pose correction for face recognition. We also compared our local appearance-based face recognition approach with two well known holistic approaches. The local appearance-based approach significantly outperformed the holistic approaches and it was more robust against the error introduced by AAM fitting and face synthesis. The second experiment evaluated the eye localization with AAM fitting. Face tracking with AAM fitting was also evaluated on a video database for open set face recognition. A modified distance from feature space metric was employed to assess the quality of fitting on a single frame. Open set face recognition was performed on the successfully registered frames. The experimental results showed that both pose correction and registration quality assessment improved the recognition performance.